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I • Executive Summary 

This document is the final report of the Working Group on Information 
Utilization and Evaluation, one of four groups formed to help define basic 
research requirements by the Renewable Resources Branch of the Earth and 
Planetary Exploration Division of NASA's Office of Space Science and Appli- 
cations (OSSA). This groups charge was to identify fundamental issues involved 
in the development of an understanding of the use of remote sensing technology 
for the acquisition data and the means by which it becomes information in the 
renewable resources management decision process. 

A primary conclusion of the Working Group is that in the second half of 
the 1980's NASA will face difficult choices among alternative fundamental and 
applied research, and development projects that could potentially lead to 
improvements in the information systems used to manage renewable resources. 

The Working Group on Information Utilization and Evaluation believes that 
effective choices cannot be made without a better understanding of the current 
and prospective problems and opportunities involved in the application of 
'•emote sensing to improve renewable research information systems. A renewable 
resources information system, as used herein, is defined in a broad context 
to include a flow of data/information from: acquisition through processing, 

storage, integration with other data, analysis, graphic presentation, decision 
making, and assessment of the affects of those decisions. An idealised re- 
source management data/information system is seen as Figure 1 herein. It is 
the Working Groups assessment that successful research and development effort 
will require some prior fundamental evaluation of both the problems which need 
solution and the alternative means by which this may be done if NASA is to 
improve the effective and efficient integration of satellite remote sensing data 
into existing and future renewable resources information systems. 
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IDEALIZED REMOTE SENSING DATA / INFORMATION FLOWS 



Figure 1. Source: Geography Remote Sensing Unit 














Therefore, the Working Group on Information Utilization and Evaluation 
recommends that NASA undertake fundamental research to: 

1. Investigate potential new technical developments 
that could significantly affect the input integra- 
tion and analysis of remotely sensed data into 
future renewable resource information systems; 

2. Develop the underlying and methodological basis 
for estimating the value of information systems 
and their modification; and, 

3. Develop after through review and assessment by an 
improved understanding of the nature of the socio- 
economic, technological and institutional linkages 
involved in existing renewable resource informa- 
tion systems, their use and users, and related 
processes . 

It is the Working Group's assessment that the research conducted in these areas 
to-date represents the weakest link in the chain of technologies required to 
yield practical social and economic benefit from advances in remote sensing. 
Advances in these areas will substantially improve the effectiveness of appli- 
cations projects and utlimately provide the basis for broad public access to 
this unique information asset. The Working Group has prioritized their re- 
search recommendations and feel that while those major areas listed first are 
more important research tasks within each of the areas seen below must be 
addressed. Every effort has been made to avoid recommending work, even if it 
is crucial, which we are certain will be satisfactorily pursued because of 
sound commercial motivations in industry. The resulting recommended areas 
for NASA sponsored fundamental research in the ares of information utilization 
and evaluation are: 

1. To conduct fundamental studies in those areas of artificial intelligence 
(AI) research which are addressing new approaches to knowledge repre- 
sentation and the associated processes of heuristic search, inference and 
learning. To pursue this research to gain an understanding of the potential 
for improving our capability to acquire, integrate, process, analyze, and 
use remotely sensed data. Initial priority subtasks here include the con- 
duct of fundamental research which: 
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A. Explores and demonstrates the linkages between: AI 

research expert systems, image analysis decision 
logic and data bases for renewable resource manage- 
ment decision making. 

B. Develops optimum methodologies involving expert 
systems to reduce requirements for sensor and ground 
truth data acquisition; and renev/able resources inven- 
tory techniques which make allowances and adjustments 
for loss of data and ultimately the availability of 
real time satellite data. 

C. Investigates expert systems to serve as a guide to 
data/information for non-expert users and for explana- 
tion of derived data (from analysis) or from conclu- 
sions obtained via remotely sensed data. 


2. To Investigate basic issues relative to the operation of geo-referenced 
data base/information systems for enhanced utilization of data/information 
derived from remote sensing data. Subtasks here include: 

A. Work towards optimal strategies for processing 
and integration of disparate data types through 
fundamental research on improving our under= 
standing of the generic functions involved in 
spatially oriented data manipulation. 

• effects of data structure on data input, 
storage, query languages, and archiving 
formats; 

• effects of positional accuracy on esti- 
mates derived from multiple data planes; 

• performance and capacity requirements for 
the large record size and special purpose 
processing required for imagery and geo- 
graphic applications; 

$ data availability, archiving and the oppor- 
tunity costs of data storage; and, 

• decentralized data base/information systems 
on cost/benefits ratios. 

B. Work with Data Base Management Systems (DBMS) research 
community (academic and industry) as they expand DBMS 

to support scientific engineering data. This work includes 
fundamental research to: 

• assist in the specification, and development 
of DBMS functions (e.g., query language ex- 
tensions) to facilitate linkage; of image and 
non-image spatial data; 

• develop prototype systems for realistic appli- 
cations; and, 

• evaluate and test products of DBMS research for 
spatial data. 
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The development of models at varying levels of aggregation of 
information systems with public, private and international com- 
ponents. In these first two areas in particular Working Group 
members felt that NASA should make every effort to keep the 
commercial computer systems as well as other private industrial 
firms in the data base/infcrmation systems and resource survey 
area informed of their needs in the area of data bases. This com- 
munication should be two-way and NASA should continue to make 
every effort to keep abrest of advances in computer sciences. 


3. To improve techniques and methodologies for documenting the accuracies of 
the products of remote sensing. There has been in the past year a sig- 
nificant literature accumulating in the reviewed literature. The Working 
Group recommends that NASA keep a close watch on the research being pre- 
sented on this topic. The Working Group still feels, however, that 
productive fundamental research here should include research which: 

A. Investigates potential tradeoffs in developing classification, 
cartographic, and mensuration accuracy for renewable resources 
applications. 

B. Examines sampling theory applied to spatial distributed, 
temporally varying, renewable resources parameters at 
varying scales. Key issues here include: 

• development of procedures with greater sen- 
sitivity to deviations from "normal conditions;" 

• identification of key parameters and correla- 
tion coefficients between image derived data 
and ground conditions for use when no dependent 
source of verification exists; and, 

• exploration of nonparametric tests of data. 

C. Develops relevant measures which characterize data/information 
(e.g., minimally sufficient statistic). 


4. To institute long range research on the theoretical basis and methodological 
approaches for estimation of the value of renewaole resources data provided 
by remote sensing. Key issues here include the: 

A. Development of a general theory of the economica of 
resource management information systems. 

B. Augumentation of current research in economic theory 
which treats information as a commodity. 

C. Examination of alternative means of assessing the public 
good and multiple use aspects of data/information. 

D. Exploring the impacts of timeliness, reliability, ac- 
curacy and assured product delivery on cost benefit 
potential . 


E. Development of improved techniques for measuring tradeoffs 
between use of existing and new information systems for 
renewable resource decision-making. 

5. To conduct an examination of both the probable and possible future environ 
ments which will impact renewable resources data bases and information 
systems. 

Finally with respect to the organization of the research proposed here the 
Working Group believes that one reason for the paucity of research in the area 
of the informations utilization and evaluation is that few scientists have 
conducted more than one study in this area. The lack of continuing involve- 
ment explains in part the absence of a general theory, methods to analyze and 
test such as theory, and empirical analysis of alternative information systems 
As a consequence, the scientific base for the design of better systems to use 
aerospace remote sensing is weak. This lack of continuity of specific tasks 
oriented support is a problem in many other areas as well. 

The Working Group, therefore, recommends that NASA support research 
to overcome these deficiencies by funding the equivalent of a "Center 
for Research on Information Systems" for a period of three to five years. 

The Center should focus on fundamental conceptual and methodological 
matters and test hypotheses relevant to information system design. The 
Center should be associated with either an academic institution or a 
private nonprofit organization rather than a government agency where 
short-term appraisals or current issues ecu Id draw resources away from 
the fundamental research effort. 

Not all research would necessarily be done at the Center. While the 
Center would be the prime contractor, some segments should be subcontracted 
to individuals and groups at other institutions with the most appropriate 
skills. Conversely, while the Center '••ould not conduct all of the research, 
it should retain substantial direct involvement and not simply organize 


- 7 - 


The Center should have an advisory committee with six to eight numbers. 

At least one third of the committee members should represent private industry. 
The remaining members should be drawn from universities and state and federal 
agencies. 

Finally, this examination of the fundamental research required to achieve 
an improved understanding of how data/information is used and valued has 
allowed Working Group members to realize that understanding existing problems 
and anticipating new ones in these areas is a difficult process requiring exper- 
tise as well as vision and foresight, thinking and rethinking to achieve a more 
meaningful appreciation of the ways in which remote sensor systemc can impact 
user needs and applications in a future time frame. We also fully realize 
that this report serves only as t small initial step in that direction. Yet, 
we also believe most strongly that a beginning must be made and we urge the 
readers of this report to pay careful attention to its summary and conclusions 
and to its recommendations. 


- 8 - 


1 1 . Introduction 

Since the launch of the first Landsat in July 1972, NASA has supported 
a considerable amount of applied research which has pointed the way to 
work still needed. For space technology to advance and to be applied to signifi- 
cant renewable resources .management, questions certain basic research problems 
remain to be addressed. As. new applications of remote sensing technology are 
made, changes become possible in data collection systems, processing tech- 
niques, and information systems. New technology cannot be developed and 
properly utilized without a better understanding of how it can contribute 
to the decision-making process. 

This document is the final report of the Working Group on Information 
Utilization and Evaluation, one of four groups formed to help define basic 
research requirements by the Renewable Resources Branch of the Resource 
Observation Division of NASA's Office of Space and Terrestrial Applications 
(OSTA). i'his group was to identify the basic issues involved in the 
development of remote sensing technology for the acquisition and applica- 
tion of information. These issues, it was agreed, had to include a con- 
sideration of decision-making processes, its context (i.e., bureaucratic, 
social, economic, environmental, and political), and the various ways data are 
transformed into information and affect decisions. Special attention was to be 
given to large renewable resources information systems (see Figure 1) - how the., 
evaluate them, by what criteria to ascertain accuracy, how to assess per- 
formance and how to anticipate future information requirements. As defined 
herein renewable resources information systems are broad in scope and em- 
ploy as one source of input data derived from satellite remote sensor systems. 

An idealized system is seen in Figure 1. In this figure the reader should 
note that elements of the system flow from the users decision as to the 
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the models he/she will employ through the selection of a sensor/platform 
to data acquisition processing storage, manipulation, integration with 
other types of data to analyze presentation, use in a decision context and 
in the monitoring of the effects of such management decisions. 

This introduction is followed by a brief background section which 
provides the reader with the context within which this report was assembled. 
Recommendation on the fundamental research required to Improve the potential 
of Satellite Remote Sensing for Renewable Resource Surveys concludes the 
body of the report. Finally, this report contains a series of Appendices 
which include added information on the Working Groups activities and delibera- 
tions. 

Appendix A contains the summary and conclusions reached by Working Group 
members in the process of this study. Appendix B provides interested readers 
with a more detailed background on the Working Groups assumptions, activities, 
and directions. Appendix C contains summaries of materials presented at the 
workshops held in the course of this study while Appendix D contains key corres- 
pondence leading to the inclusion of specific material. Appendix E lists 
individual Planning Session and Workshop participants and summarizes affiliations 
while the final Appendix F contains the visuals employed at the briefing for 
NASA Headquarters personnel in May 1981. 


Ill . Background 

The future potential of aerospace remote sensor systems is truly 
extraordinary. The opportunities they provide, however, will require 
public decisions concerning satellite sensors, ground facilities, and 
analysis systems of great complexity and cost. While the potential bene- 
fits of future systems are large, there are data management and storage 
problems. Various systems could provide information bearing on important 
national and global resource planning and management concerns, but the 
legal, social, and economic problems which they create are of national 
and international significance. 

OSSA currently has a number of applied research and development 
projects (e.g., AgRISTARS) which apply existing knowledge to design, 
engineer and demonstrate the capabilities of aerospace remote sensing 
as a resource management information source. The Fundamental Research 
Ingram is intended to complement these projects by focusing on concepts 
and issues at the frontiers of the relevant sciences. A successful pro- 
gram will provide essential building blocks for future applied projects 
to support the design of sensors, ground facilities, and institutions to 
facilitate national and possibly increased international use of remote 
sensing systems in the late 1980's and 1990's. The research will be 
conducted by scientists who are familiar with current and potential appli- 
cations of aerospace remote sensing technology to renewable resources. 
Research results are expected to be communicated worldwide by the publi- 
cation of papers in recognized scientific journals and other types of 
appropriate publications. 
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The Program Definition Study of Basic Research Requirements in 
remote sensing is divided into four general areas of study: 

(1) Scene Radiation and Atmospheric Effects Characterization; 

(2) Pattern Recognition and Image Analysis; 

(3) Electromagnetic Measurements and Data Handling; and, 

(4) Information Utilization and Evaluation. 

Working groups composed of scientists and practitioners from universities, 
research Institutions, industry and government have been established for 
each research area. Members were familiar with potential applications 
and with the problems associated with information from remotely sensed 
data. These groups have planned and conducted workshops where qualified 
experts could hear and discuss one another's views concerning the current 
state of knowledge, on-going work at the frontiers, and areas where 
research could produce significant results. The workshops provided 
opportunities for groups to draw broadly on the knowledge of the 
scientific community and deepen their understanding of relevant issues 
before preparing recommendations to NASA. 

Each working group is to develop a prioritized list of research topics 
and a preferred method for implementing and coordinating its recommended 
program of basic research. Each group leader is supported by selected 
members of the working group who are responsible for documenting findings 
and recommendations in a report to NASA. These reports will help NASA 
define a three-to-f ive year research program to broaden and strengthen 
the scientific base for future applied research and development programs 
directed at better use of aerospace and remote sensing in monitoring the 
Earth's renewable resources. 
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The Working Group on Information Utilization and Evaluation organized 

and held its initial planning session in April, 1980 at Santa Barbara, 

California. Members of the group include: 

Glen Bacon, IBM Corporation 

Nevin Bryant, NASA Jet Propulsion Laboratory 

Christopher Clayton, University of California 

Ludwig Eisgruber, Oregon State University 

John Estes, University of California 

Forrest Hall, NASA Johnson Space Center 

Ida Hoos, Space Science Laboratory, University of California 

Robert MacDonald, NASA Johnson Space Center 

Bruc.e Scheer, The Planning Economic Group, Boston 

Ronald Shelton, Michigan State University (deceased) 

Charles Vars, Oregon State University 

a second planning meeting was held in July 1930 at Johnson Space Center 

to: (l) identify broad areas in which basic research is appropriate and 

(2) plan workshops to address these research needs. (Participants in 

each meeting, as well as workshop, are listed in Appendix 0. 

The Working Group on Information Utilization and Evaluation conducted 
three workshops. The first workshop on Information and Decision Processes 
was held at Asilomar, California in September, 1980; the second workshop 
on Data Base Use and Management at San Jose, California in January 1981; 
and the third workshop on Data and Information Performance at Houston, 

Texas in February 1981. The Working Group also held a special session at 
the conference on Remote Sensing for Resource Management sponsored by 
the Soil Conservation Society of America at Kansas City in October 1981. 
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I V • Research Recommendations 

Introduction . In the second half of the 1980' a NASA will face dif- 
ficult choices among alternative applied research and development projects 
that could potentially lead to improvements In the information systems 
used to manage renewable resources. The Working Group on Information 
Utilization and Evaluation believes that effective choices cannot be made 
without a better understanding of the current and prospective problems 
and opportunities to improve renewable resource information systems. 

A successful research and development effort will require prior fundamental 
evaluation of both the problems which need solution and the alternative 
means by which this may be done. 

Therefore, the Working Group recommends that NASA undertake fundamental 
research to: 

1. Investigate potential new technical developments that could 
significantly affect the input integration and analysis of 
remotely sensed data into future renewable resource infor- 
mation systems; 

2. Develop the underlying theoretical and methodological basis 
for estimating the value of information systems and their 
modification; and, 

3. Develop after thorough review and assessment an improved 
understanding of the nature of the socioeconomic, technological 
and institutional linkages involved in existing renewable re- 
source information systems, their use and users and related 
processes . 

The premises underlying these recommendations are quite simple. 
Effective choices among future NASA R S D projects can only be made once 
the agency understands the problems and opportunities to improve renewable 
resource information systems. When contemplating R & D decisions, the 
first step must be to determine whether and why there are problems. 

Relevant questions include: where, why, and to what extent do systems 

fail and succeed today? Are systems likely to perform differently in the 
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future, and if so, in what respects? Are the gaps between actual and 
"ideal" performance likely to be persistent and important, or only trans- 
itory and relatively unimportant? These questions should be addressed and 
answered by research in the second and third areas noted above. 

While assessing the nature of the research tasks to be accomplished 
to help find solutions to these problems, NASA should identify further 
developments in the capabilities of remote sensing systems that could 
improve overall renewable resources information system performance. In the 
view of the Working Group the alternatives to be investigated should en- 
compass a broad range of institutional as well as technical developments. 
Examples include the examination of probable and potential operating 
environments and information needs in the 1990's, as well as studies of 
strategies for processing disparate types of data, potential linkages 
between artificial intelligence oriented "expert systems," and improved 
techniques to assess the accuracies and other performance characteristics 
of remote sensing systems. Of course, new developments in data handling 
and management systems also deserve careful attention. 

Ultimately, however, NASA will nave to value the consequences of 
alternative projects and actions to modify existing or potential remote 
sensing components of renewable resources information systems. Such 
valuations will be difficult but necessary if NASA is to select the "best" 
projects and actions. The major questions to be answered are: How should 
information system performance be measured and evaluated? How should 
different combinations of improvements in system performance be compared 
with one another? What are the impacts of changes in timeliness, relia- 
bility, accuracy, and assured product delivery? Clearly some valuation 
problems may not lend themselves to quantification. The choice among 
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alternatives will never be easy, for systems are comolex, the future 
is uncertain, and tradeoffs are inescapable. Nonetheless, the Working 
Group believes that a long-range research effort to develop a better 
basis for valuing modifications in information systems can contribute 
to the design of an increasingly effective R & D program for NASA. 

The preceding discussion has described the broad fundamental research 
recommendations of the Working Group on Information Utilization and Eval- 
uation. The following materials describe the recommended research topics 
more specifically. Neither the discussion nor the listings of research 
tasks should be considered all-inclusive. Rather they represent a dis- 
tillation of the presentations at workshops, conversations with knowledge- 
able individuals, and discussions among the Working Group. The appendices 
provide a more complete description of the many research topics identified 
during the project. What follows, however, is a prioritized ranking of 
the most important research topics identified by the Working Group for 
recommendation to NASA. 

The following, ^“commended research topics, then, represent a dis- 
tillation of the discussions held by information utilization and evaluation 
working group members over the course of this project. The material sifted 
in the preparation of the recommendations presented herein was large in 
volume and broad in scope. The recommendations 1 isted below represents our 
best efforts to encapsulate material pertinent to NASA goals and missions 
which were presented in planning meetings, workshops, conversations with 
knowledge base of individual working group members themselves. In these 
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recommendations we have attempted to carefully screen those areas of pri- 
vate sector and academic research efforts where NASA can derive real bene- 


fits from following research in progress from those where work is required 
and appropriate for NASA to find. The following five research recommenda- 
tions are ranked in priority order. Within each of these major recommended 
research areas individual tasks are also listed in order of their priority. 


Working Group members feel strongly that NASA should do its best to imple- 
ment programs of fundamental research directed at as many of the topics 
listed below as practical. Recommended research areas are the following: 


1. To conduct fundamental studies in those areas of artificial intelli- 
gence (AI) research which are addressing new approaches to knowledge 
representation and the associated processes of heuristic search, in- 
ference and learning. To pursue this research to gain an under- 
standing of the potential for improving our capability to acquire, 
integrate, process, analyze, and use remotely sensed data. 


This recommendation involves the initiation 


of 


limited number of key 


applications oriented fundamental research programs. The committee feels 
that it is from within the context of such decision oriented research 


efforts that an improved understanding of the basic nature of the resource 
management decision process will flow. We feel that such understanding 
is essential if we are to strengthen the weakest link in the chain of the 
use of remotely sensed data (i.e., its employment in a management decision 
context). Some basic characteristics of "expert systems" that involve 
levels of systems, processing, analysis and human factors considerations 
crucial for ultimate acceptance of information generated are that: 

1) They include a data base of information which is directly 
related to a given problem. This data base can be of 
many types and have input from a variety of sources, in- 
cluding the judgemental knowledge of expert in a given 
area on the probability of certain events or phenomena 
begin related or happening given a defined set of con- 
ditions; 

2) Such systems should be able to explain their decisions 
in terms of a line of reasoning that the ultimate user 
of the system can understand; 
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3) They should be capable of justifying their performance 
by responding to questions expressed in plain simple 
language; 

4) These systems should be able to "learn" new informa- 
tion quickly by Interacting directly with experts in 
their given area; 

5) Their knowledge base must be easy to modify so the 
perceived errors can be corrected rapidly before they 
recur in another case; and, 

6) Interaction should be designed with the user in mind 
(i.e., programmed in terms of prompts, answers and in- 
formation volunteered by the system as well as by the 
user) . 

Research exploring the linkage of such systems in the remote sensing 
geographic information system resource management decision process is needed 
in the following areas: 

Discipline Concepts - Traditional image analysis systems have no true 
"understanding" of the discipline involved. Although explicit decision trees 
can make programs more relevant, true discipline knowledge and the heuristics 
for problem solving are neither explicitly represented nor used. In fact, 
"commonsense" is often absent when existing systems fail, and this justifiably 
alienates users. In contrast, expert systems nwke explicit use of produc- 
tion rules that relate observations to inferences. Research is required 
to move these systems into the image analysis chain. 

Conversational Capabilities - Research is needed to develop computer- 
based linguistic capabilities for capturing knowledge from collaborating 
experts and for communicating with users in terms that have clear and 
concise meaning to them. 

Explanation - Resource management decision systems currently providing 
information to users seldom explain the basis for decisions made with 
respect to a given product or display in terms understandable to users. This 
can mean the user depending upon his own knowledge base can either have 
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excellent, if he truely understands the system, or little or no basis for 
deciding whether to accept its results. In contrast, the heuristics of an 
expert system could provide an explanation of the underlying reasoning. 
Research here is reuqired to begin to meaningfully demonstrate the poten- 
tial of those systems. 


1. Subtask Expert Systems Fundamental Research .opics Include: 

A. Exploration and demonstration potential Iv.Vages between 
artificial intelligence oriented "expert systems," image 
analysis logic, and data bases for resource ivnagement 
decision-making. Within the management procef as mater 
i a 1 on which decisions are based becomes more complex, 
decision processes become less deterministic and more 
heuristic. Because of this the committee foresees the 
potential of a highly productive fundamental research 
effort aimed at exploiting expert systems within a 
resource management context. 

B. Develops optimum methodologies involving expert systems 
to reduce requirements for sensor and ground truth data 
acquisition; and renewable resources inventory tech- 
niques which make allowances and adjustments for loss 
of data and ultimately the availability of real time 
satellite data. As a variety of data types exist on 
many environmental parameters methodologies for the 
incorporation and assessment of the utility of these 
data could prove very helpful in identifying key gaps 
in existing systems. Such information might also be 
used to identify those gaps likely to persist through 

a considerable time period. 

C. Investigates expert systems to serve as a guide to 
data/information for non-expert users and for explana- 
tion of derived data (from analysis) or from conclu- 
sions obtained via remotely sensed data. Inherent 

in the use of expert systems is the development of 
"user friendly" interfaces with the systems being 
employed. This includes the development of improved 
data base management systems as discussed in area 2 
below. 
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2. Investigate basic Issues relative to the operation of geo-referenced 
data base/information systems for enhanced utilization of data/ 
information derived from remote sensing data. Subtasks here include: 

A. Work towards optimal strategies for processing and inte- 
gration of disparate data types through fundamental re- 
search on improving our understanding of the generic 
functions involved in spatially oriented data manipula- 
tion. Some issues here include the: 

• effects of data structure on data input, storage, 
query languages, and archiving formats. Alterna- 
tive methods for optimizing SDatially-related 
(e.g., geographically ordered relational, or 
modified hierarchical ) data base structures need 
to be examined in light of the requirements for 
capture, storage, integration, manipulation and 
access access a multiplicity geo- referenced data 
planes. 

• effects of positional accuracy on estimate derived 
from multiple data planes. All mapping involves 
global and local positioning from the analysis of 
several mapping phenomena, errors are cumulative. 
Investigations need to be undertaken to determine: 
a) the sensitivity of models to positioning 
errors in input data sets; b) the potential for 
satellite data to improve the integration of spa- 
tial phenomena in functions such as trend surface 
mapping and development of multi-stage sampling 
designs. 

• performance and capacity requirements for the large 
record size and special purpose processing required 
for imagery and geographic applications. Factors 
such as timeliness and repetitiveness of satellite 
overflights will continue to be essential drivers of 
satellite systems. All stages in ground data acqui- 
sition, calibration, geocoding, archiving, transfer- 
ing, and analysis need to meet the satellite over- 
flight requirements if an end product is to be pro- 
vided. Given certain assumptions in future analysis 
requirements, what are the key stages in data 
preparation which need to have improved through-put? 

How would each stage in data preparation and analysis 
best be served ( i . e . , centralized or decentralized 
function)? Are basic structural changes implied if 
operational capability is to be achieved? 

• data availability, archiving and the opportunity 
costs of data storage. Costs of storage and archiv- 
ing are decreasing, however, the volume of data 
produced by remote sensor systems is high. We need 
to examine the large range implications of data loss 
on renewable resource surveys. Other areas here 
include universal cataloging systems for a variety of 
satellite data rather than the ad hoc procedures de- 
veloped to date; and, 
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• decentralized data bases/lnfornation systems on cost/ 
benefits ratios.. Do field users have need to combine 
there specific data bases with a common minimal data 
base? What is an optimal decentralized data base 
structure to allow rapid and efficient access by re- 
mote users of renewable resources data. 

B. Work with the Data Management Systems (DBMS) Research Community 
(academic and industry as thev expand DBMS) to support scientific 
engineering data. This shouk’ include the development of proto- 
type DBMS's which link user friendly expert systems and the large 
storage and file structure systems presently required to store, 
integrate and manipulate the spatial data sets which are derived 
from remotely sensed and other renewable resources survey data 
sources. To do so will require a program of development test- 
ing and evaluation of appropriate DBMS's for spatial data. 

C. Develop models at varying levels of aggregation of resource 
information systems with public, private and international com- 
ponents. As data are derived from sources at a variety o e 
levels are aggregated and disaggregated in the decision ptocess 
misrepresentation can occur. What is called for is an effort 
to understand the nature and magnitude of the potential errors 
inherent in this process. 

A number of the task sub-areas, discussed above, are based upon con- 
clusions reached at the San Jose Workshop on Data Base Management and Use. 

In particular working members felt that it is imperative for NASA to make 
every attempt to keep the Commercial Computer Systems industry informed 
of their needs in tie area of data bases. This communication should be 
a two-way flow, however, and NASA Earth and Planetary Exploration Division 
should make a concerted effort to keep abreast of the rapid advances in 
computer science so as to better integrate the potential of new develop- 
ments into future satellite sensor systems. The Working Group believes 
then that NASA should work with the data base management systems community 
(academic and industry) to support research on the linkage, management, 
and use of image and non-image spatial data. The research community 
is currently working to improve our ability to manage large data files 
and perform special purpose processing. Major breakthroughs will be forth- 
coming, but they will address the problems associated with large spatial 


data sets only if NASA commml cates its long-range needs to the research 
community. 
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3. Conduct fundamental analyses of the question of techniques and methodo- 
logies of documenting the accuracies of the products of remote sensing. 

This area also includes a research issue for the fundamental research 


program area studying Data Handling and Measurement Systems. That is, the 
development of procedures necessary for both present and future renewable 
resources satellite sensor systems to approach "mensuration" system status 
for given applications. Research here include: 


A. Investigate the impacts on sensor systems, processing and 
analysis techniques to the development of standards for 
classification, clarification and mensuration accuracy of 
remotely sensed data. It is important to note here that 
in the area of thematic accuracy assessment there is a con- 
siderable amount of material beginning tc accumulate in the 
reviewed journal literature. It is apparent that this is 
an area of current attention by a number of individuals both 
with government and academia in the United States and abroad. 
NASA may not need to fund work herein, but should realize that 
this work may have profound implications on future satellite 
sensor systems development. 


B. Continue to examine the fundamental aspects of sampling 
theory as they apply to spatial distributed, temporally 
varying, renewable resources parameters at local, regional, 
national, and international scales. Key research issues 
here include: 

• development of procedures which exhibit a greater 
sensitivity to deviations from "normal conditions;" 

• identification of key parameters and determination of 
correlation coefficients between remote sensor scene 
derived data and ground conditions for use when on 
independent source of verification exists; and, 

• the exploration of the potential of nonparametric 
tests of data. 


C. Development of relevant measures which characterize data/infor- 
mation (e.g., Minimally Sufficient Statistic). 


For many types of renewable resource survey problems classical statis- 


tical approaches are "nearly applicable." Nearly applicable as most of the 
assumptions upon which classical statistical approaches are based are vio- 
lated in their application to specific resource inventory problems. 
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What Is required then is to examine the potential for fundamental research 
directed toward the development of new statistical sampling strategies which 
are more robust and sensitive to environmental variation. This is an impor- 
tant area for fundamental research as sampling is still a major concern and 
yet a number of f he participants in this study also felt that methods for 
achieving improved verification of local identification and classification 
employing non-sampling approaches are also important and certainly needed. 

4. Institute a long-range program to provide a better understanding of the 
theoretical bases and methodological approaches capable of being 
employed for estimation of the value of renewable resources data pro- 
duced by remotely sensed data. 

Research tasks in this area would include: 

A. Development of a general theory of the economics of 
resource management information systems. 

B. Augumentation of current research in economic theory 
which treats information as a commodity. 

C. Examination of alternative means of assessing the public 
good and multiple use aspects of data/information. 

D. Development of improved techniques for measuring trade- 
offs between use of existing and new information systems 
for renewable resource decision-making. 

E. Investigation of the impacts of timeliness, reliability, 
accuracy and assured product delivery on cost benefit 
potential . 

The Working Group finds that there is no generally accepted framework 
for estimating the value of modifications in public sector components of 
information systems. A number of factors contribute to this methodological 
problem. First, there is no market price for the output of most public 
information systems, and therefore value cannot be estimated in conventional 
ways. Second, the quality of an information system is based on its multiple 


characteristics, including accuracy, timeliness, reliability, continuity, 
and so on. Moreover, some types of information possess the character- 
istics of a public good (the use of the good by one individual does not 
reduce the amount available to others), and its private value may differ 
substantially from its social value. Third, many sources of information 
are available to decision-makers, few are regarded as certain, and choices 
among types of information to obtain and use are made within uncertain 
environments that confront decision-makers. Fourth, conceptual problems 
recur for information systems because of changes in reality and the agenda 
for decisions. Learning, development, and innovation are common, and 
information systems evolve. 

Some good work has been done to address each of these problems. None- 
theless, only rudimentary models have been developed, tested, and empir- 
ically estimated. Several methodological approaches have been used, but 
controversy continues concerning the appropriate methodology. No method- 
ology has yet proven successful in general applications; in fact, dif- 
ferent applications raise more questions than they have answered concerning 
the appropriate methodology. 

The Working Group believes that a high priority should be assigned 
to the development of the underlying theoretical and methodological basis 
for estimating the value of information systems and their modification. 

The goal would be the formulation of quantitative models capable of con- 
tributing to the evaluation of trade-offs between existing and potential 
new information systems. Such models would be able to describe and 
explain existing systems and to provide the means to identify and estimate 
f he magnitude of future consequences of new systems with alternative 


features. 
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One methodology which the Working Group recommends for addressing 
these issues involves the review and assessment of existing information 
systems. Existing information systems, their use and users, and related 
processes need to be reviewed and their performance assessed before poten- 
tial future systems can be evaluated. The objective should be the review 
and documentation of multi-purpose, government and private systems with 
international dimensions. It is not feasible to analyze all systems where 
remotely sensed data are currently being or potentially could be used. 
Therefore, research should focus on selected information systems where 
remotely sensed data are or may be combined with data collected by other 
means. 

Information systems develop to help decision-makers make decisions 
to solve problems. Since problem-solving requires some understanding of 

the nature of the problem to be solved, information must be viewed as the 

product of some process of inquiry. From this perspective, knowledge 

about a problem is dependent on the system of inquiry used to obtain that 

knowledge. To contribute to solving problems, an information system 

represents the reality relevant to those problems and the decisions to be 

made. As a .consequence, the goals and values of the decision-maker affect 

the design of an information system, and the decision-maker must be 

considered as a major component of the information system. 

Most studies on the value of information assume, for example, that 
a farmer, an elevator operator, and a buyer all use the same information 
in the same way, but evidence indicates this is unrealistic. Some infor- 
mation is available as a public good, but it is not equally used by all 
who receive it. For these reasons, the identification of major market 
and nonmarket users of information, their sources of information, and the 
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adequacy of these sources become Important if new systems are to be satis- 
factorily evaluated. Particular attention should be given to the documen- 
tation of how the behavior of public and private users and producers of 
information are changed when information is viewed as a public rather than 
a private good. Data and information dissemination systems need to be 
analyzed. These studies should examine the accessibility, decentralization, 
economies of scale, confidentiality, and property rights of existing systems. 

Information systems have multiple components and data sources because 
decision-makers seek different types of information on the various "fields 
of reality" with which they must deal. Decision- and policy-makers typically 
utilize data and information with a wide variety of attributes. Attributes 
such as function, scale, timeliness, precision, and others should be examined 
in the context of systems where tradeoffs among attributes and institutional 
constraints can be directly addressed. 

Systematic reviews and documentation are also needed to understand 
how the attributes of data and information derived from it interact and 
affect analytical and decision processes. These investigations should 
document analytical systems and processing strategies that extract infor- 
mation for decision-making from data based on disparate concepts. Special 
attention should be given to why, how, and with what effects are data from 
multiple sources used within existing information systems. 

Information systems are used in both an ordinary and extraordinary 
mode, and the reviews of existing systems should not focus on one mode 
of use and exclude the other. In the ordinary mode of use, information 
accords reasonably well with expectations and only routine adaptations 
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are called for within the operating environment. In ordinary circumstances 
recurrent mismatches between observations and expectations do not occur, 
there is no need to change the information system, and a review will 
reveal the strengths of the system. In the extraordinary mode, however, 
information does not accord with expectations, and non-routine actions 
are taken. Learning, development, and innovation are then initiated to 
transform the information system. These processes are not well under- 
stood, in part because existing systems have not been thoroughly reviewed 
and documented. 

The final research area recommended by the Working Group on Information 
Utilization and Evaluation is: 

5. Conduct an examination of both the probable and possible future environ- 
ments which will impace renewable resources data bases and information 
systems. 

This recommendation differs from general recommdation #4 in that more 
emphasis in this area would be put towards examination of future environments 
rather than beginning with existing systems as is proposed in #4. 

What has been stressed is a better understanding of the user management 
decision process. Throughout this document is the need to achieve acceptance 
of the basic reality that remote sensing is not, in and of itself, a unique 
information source. It is, however, when used properly a very powerful 
multi-dimensional data source. Yet, remote sensing will only realize its full 
information potential when data produced by remote sensor systems is effec- 
tively and efficiently integrated with data from other more "conventional" 
sources in the management decision process. This requires that we improve 
at a fundamental level our understanding of not onl> the management decision 
process but of the type and characteristics of the data bases employed by 
resource management decision makers as well. For in "management environments. 
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where cost effectiveness Is the inflexible justification for present and 
especially future generations of satellite technology (with reference to 
value and returns), one finds it at present somewhat comparable to quantifying 
pie-in-the-sky . It is not easy tc complie credible numbers to satisfy con- 
gressional policy makers. Nor is it an easy task with the traditionally 
axe wielding 0MB. What we can expect from not only the executive but even 
from the public at large is to some extens already the handwriting on the 
wall. In an era of Federal fiscal restraint, it is unrealistic to count on 
bit investments and quixotic to make claims that cannot be viholly substantiated. 
Technology cannot be rationalized by non-existent demand. The public has 
its own perceptions of pundits and in the present political climate not all 
premises carry the same amount of credibility. 

This brings us back into the morass of factors impeding proper assess- 
ment of the use and value of information with respect to the resource manage- 
ment decision process. It also, in all likelihood, explains the dearth of 
such, evaluations. The social context within which such evaluations must 
occur today renders them mutually impossible. But, is this to say then that 
none should be done? Quite the contrary. Starting with the premise that 
the social context crucial, we might do './ell to consider another socval 
context — not the one that serves as a kind of gridlock but one that provides 
us with what seems to be a generally agreed upon paradigm for the future and 
from which a series of scenarios could be extracted to serve as drivers for 
pursuing fundamental investigations of the nature of mangement decision 
data bases and their impacts upon the decision process. The new world of the 
future as envisioned in the Global 2000 report might be a good framework. 


It is the global situation needs of the year 2000, and on, to which tomorrow's 
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technology must respond, and it is clear that remote sensing and the infor- 
mation it can provide could be a key factor in meeting the challenges which 
transcend all national borders and internicine warfare among government 
agencies at all levels and even the commercial sector as well. Because it 
is the far horizon that has more certainty and for which there is a greater agree- 
ment than the present, the information utilization and evaluation working 
group feels that NASA should consider it as the proper framework for guiding 
the scenarios suggested here, and thus unfetter ourselves with preoccupation with 
fabricating a story designed to please rather than to enlighten. 

Finally, with respect to the organization of the research proposed 
here the Working Group believes that one reason for the paucity of research 
in the area of the economics of informations sytems is that few scientists 
have conducted more than one study in this area. The lack of continuing 
involvement explains in part the absence of a general theory, methods to 
analyze and test such as theory, and empirical analyses of alternative 
information systems. As a consequence, the scientific base for the de- 
sign of better systems to use aerospace remote sensing is weak. This 
lack of continuity of specific tasks oriented support is a problem in 
many other areas as well. 

The Working Group, therefore, recommends that NASA support research 
to overcome these deficiencies by funding the equivalent of a "Center 
for Research on Information Systems" for a period of three tr> five years. 

The Center should focus on fundamental conceptual and methodological 
matters and test hypotheses relevant to information system design. The 
Center should be associated with either an academic institution or a 
private nonprofit organization rather than a government agency where 
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short- term appraisals or current issues could draw resources away from 
the fundamental research effort. 

Not all research would necessarily be done at the Center. While the 
Center would be the prime contractor, seme segments should be subcontracted 
to indi vidua! s and groups at other institutions with the most appropriate 
skills. Conversely, while the Center would not conduct all of the research, 
it should retain substantial direct involvement and not simply organize 

the research. (The Center could become the lead institution in a consor- 
tium which conducts research over a longer term and larger scale than 
that by NASA). This approach would assure a focus of responsibility, 
a- we 11 as coordination, continuity and visibility. 

The Center should have an advisory committee with six to eight 
numbers. At least one third of the committee members should represent 
private industry. The remaining members should be drawn from univer- 
sities and state and federal agencies. 
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